Mental Constitutions and Limited Rationality ( Extended Abstract )

One attractive route to developing a theory of limited rationality is through the notion of rational control of reasoning. Rational control is sometimes viewed in terms of rational allocation of limited computational resources in drawing conclusions, with ideal rationality just the infinite-resource case of resourcebounded inference. But some important sorts of limitations on rationality due to rational control of reasoning subsist independently of any limitations on resources. One of these is the notion of the agent’s constitution, a set of restrictions placed on the permissible states of the agent by the agent or by its designer. While the theory of ideal rationality permits agents to have any consistent and complete set of beliefs and preferences (probabilities and utilities), agents limited by constitutions may forbid some of those sets. We motivate and formalize some elements of a theory of mental constitutions. Introduction: Limits to rationality Artificial intelligence is concerned with agents whose rationality is limited in comparision with the agents presumed by decision theory. In decision theory, a rational agent bases its actions on its beliefs about the relative likelihoods of various events and contingencies and its preferences among these. It requires that these beliefs and preferences are consistent and complete enough to determine probability and utility measures. The limitations accepted as unavoidable in artificial intelligence include having to work with less information (more incomplete, possibly inconsistent sets of beliefs and preferences) and with bounds on the available computational resources. Such limitations have led to theories of resource-bounded reasoning in which only some of the consequences of basic ∗Research supported by National Institutes of Health Grant No. R01 LM04493 from the National Library of Medicine. c © 1989 by Jon Doyle beliefs are computed when choosing actions. It is natural to view these theories as cases of rational control of reasoning, in which the agent rationally allocates its resources in each step or episode of reasoning. One conception of rational control of reasoning views the task of reasoning as that of computing a sufficiently complete and consistent set of beliefs and preferences from those manifest in memory. In this case, the ideally rational agent is just an agent with unbounded resources available to compute an entire complete and consistent set of attitudes. Limited agents, in contrast, will compute sets of attitudes which are complete in only some ways, and which may contain some sorts of implicit inconsistencies. In some cases, we may view these partially complete and consistent sets of attitudes as sets complete and consistent with respect to a weaker than ordinary logic (e.g., as does Konolige [1985]). But this conception of resource-bounded reasoning does not provide an entirely adequate theory of limited rationality by itself, as some important sorts of limitations on rationality subsist independently of any limitations on resources. Some of these limitations, such as the apparent impossibility of reconciling multiple preference criteria in default reasoning shown by Doyle and Wellman [1989], stem from our wish to facilitate the construction of complex individual agents by decomposing them into sets of simpler “mental agents” or faculties. Another sort of decomposition is that involved in viewing the agent’s mental states as sets of attitudes (e.g., beliefs, preferences, intentions). This decomposition leads to more resourceindependent limitations, namely restrictions placed on the permissible states of the agent by the agent or by its designer. For example, states might be forbidden to contain certain consistent beliefs or combinations of beliefs, or might be required to contain other beliefs. Such restrictions do not violate any axiom of decision theory by themselves, since decision theory does not require that all consistent and complete sets of beliefs and preferences be acceptable. But these restrictions would seem to violate the spirit of decision theory, whose presumption that all consistent and complete sets of attitudes be possible is clearly visible in much of